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[0010] 

As described above, CMOS sensors fiave been used In 
portable apparatuses, with their characteristics of smaller sizes and 
lower power consumption. On the other hand, they have a serious 
problem of degradation of images due to hand-shake, in cases of 
portable image inputting apparatuses. 
[0011] 

For example, with reference to Fig. 7(a), in cases where a 
rectangular object is photographed with an image inputting 
apparatus, if the image inputting apparatus is at rest and no 
hand-shake occurs, no abnormality will occur in a captured image. 
[0012] 

On the contrary, if the image inputting apparatus is shaken by 
hand In the leftward direction, with reference to Fig. 7(b), and If the 
image inputting apparatus is shaken by hand in the rightward 
direction, with reference to Fig. 7(c), large distortions will occur in 
the captured image. 
[0013] 

With reference to Fig. 7(d), if the image Inputting apparatus is 
shaken by hand in the upward direction, the object is photographed 
in a vertically-enlarged manner. Further, with reference to Fig. 7(e), 
if the image inputting apparatus is shaken by hand in the downward 
direction, the object is photographed in a vertically-contracted 
manner. 
[0014] 

Particularly, image distortions caused by rightward and 
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leftward hand-shakes will significantly change the captured shape of 
the object, and therefore, will be significantly visually obtrusive to 
cause image degradations. 
[0015] 

Accordingly, there is necessarily a need for a countermeasure 
against these phenomena. As such a countermeasure, there has 
been suggested in JP-A No. 9-181986, a method for rapidly reading 
out images from a solid image pickup device within a time interval 
smaller than a single-frame time interval. However, this method 
increases the clock frequency required for readout, thereby causing 
the problem of an increase of power consumption. 
[0016] 

The present invention is made in view of the aforementioned 
circumstances and alms at providing an image inputting apparatus 
capable of correcting image distortions caused by hand-shakes 
without increasing power consumption, in cases of using an X-Y 
address type solid image pickup device. 
[0017] 

[Means for Solving the Problem] 

An image Inputting apparatus according to an aspect of the 
present invention includes a solid image pickup device having an 
area sensor portion including photoelectric conversion devices 
arranged on a two-dimensional plane defined by predetermined first 
and second directions, a first scanning portion which is connected to 
the area sensor portion and selects in order in the first direction the 
respective lines of the photoelectric conversion devices in the 
second direction in the area sensor portion on a line-by-line basis 
and a second scanning portion which is connected to the area sensor 
portion and selects in order in the second direction image-pickup 
signals output from the photoelectric conversion devices along the 
lines; a hand-shake detecting portion for detecting hand-shake 
generated during photographing and outputting hand-shake 
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information according to tlie direction and the amount of the 
hand-shal<e; a hand-shal<e correction value calculating portion 
which is connected to the hand-shake detecting portion and 
calculates the amount of movement of the solid image pickup device 
in the second direction, on the basis of the hand-shake information 
output from the hand-shake detecting portion; and a timing" 
generating portion which is connected to the hand-shake correction 
value calculating portion and the solid image pickup device and 
controls to change the pixel selection starting position for the 
second scanning portion, on the basis of the amount of movement In 
the second direction and the line position selected by the first 
scanning portion. 
[0018] 

The timing generating portion changes the pixel selection 
starting position for the second scanning portion, on the basis of the 
amount of movement in the second direction and the line position 
selected by the first scanning portion. This enables correcting the 
image distortions caused by the hand-shakes in the second 
direction. Further, image-pickup signals can be read out from the 
solid Image pickup device using a conventional clock frequency, 
thereby preventing increase of power consumption. 
[0019] 

Preferably, the hand-shake correction value calculating 
portion includes a first-direction hand-shake correction value 
calculating means for calculating the amount of movement of the 
solid image pickup device in the first direction, on the basis of the 
hand-shake information output from the hand-shake correction 
portion and a second-direction hand-shake correction value 
calculating means for calculating the amount of movement of the 
solid image pickup device in the second direction, on the basis of the 
hand-shake Information output from the hand-shake correction 
portion. The image inputting apparatus further includes an image 
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storage portion which is connected to the solid image pickup device 
and stores image-pickup signals read out from the solid image 
pickup device, corresponding to at least a single screen; and an 
image storage control portion which is connected to the 
first-direction hand-shake correction value calculating means and 
the image storage portion and controls the readout in the first 
direction of image pickup signals stored in the image storage portion, 
on the basis of the amount of movement in the first direction. 
[0020] 

The image storage control portion controls the readout in the 
first direction of image pickup signals stored in the image storage 
portion, on the basis of the amount of movement in the first 
direction. This enables correcting the image distortions caused by 
hand-shakes in the first direction. 
[0021] 

An image inputting apparatus according to another aspect of 
the present invention includes a solid image pickup device having an 
area sensor portion including photoelectric conversion devices 
arranged on a two-dimensional plane defined by predetermined first 
and second directions, a first scanning portion which is connected to 
the area sensor portion and selects in order in the first direction the 
respective lines of the photoelectric conversion devices in the 
second direction in the area sensor portion on a line-by-line basis, 
and a second scanning portion which is connected to the area sensor 
portion and selects in order in the second direction image-pickup 
signals output from the photoelectric conversion devices along the 
lines; an image storage portion which is connected to the solid 
image pickup device and stores image pickup signals read out from 
the solid image pickup device, corresponding to at least a single 
screen; a hand-shake detecting portion for detecting hand-shake 
generated during photographing and outputting hand-shake 
information according to the direction and the amount of the 
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hand-shake; a hand-shake correction value calculating portion 
which is connected to the hand-shake detecting portion and 
calculates the amount of nnovement of the solid image pickup device 
in the second direction, on the basis of the hand-shake information 
output from the hand-shake detecting portion; and an image 
storage control portion which is connected to the hand-shake " 
correction value calculating portion and the image storage portion 
and controls the readout in the second direction of image-pickup 
signals stored in the image storage portion, on the basis of the 
amount of movement in the second direction. 
[0022] 

The image storage control portion controls the readout In the 
second direction of Image pickup signals stored in the image storage 
portion, on the basis of the amount of movement in the second 
direction. This enables correcting the image distortions caused by 
hand-shakes in the second direction. Further, image pickup signals 
can be read out from the solid Image pickup device using a 
conventional clock frequency, thereby preventing Increase of power 
consumption. 
[0023] 

Preferably, the hand-shake correction value calculating 
portion includes a first-direction hand-shake correction value 
calculating means for calculating the amount of movement of the 
solid image pickup device in the first direction on the basis of the 
hand-shake information output from the hand-shake correction 
portion, and a second-direction hand-shake correction value 
calculating means for calculating the amount of movement of the 
solid image pickup device in the second direction, on the basis of the 
hand-shake information output from the hand-shake correction 
portion. The image storage control portion includes a 
first-direction image storage control means which is connected to 
the first-direction hand-shake correction value calculating means 
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and the image storage portion and controls the readout in the first 
direction of image pickup signals stored in the image storage portion, 
on the basis of the amount of movement in the first direction, and a 
second-direction image storage control means which is connected to 
the second-direction hand-shake correction value calculating means 
and the image storage portion and controls the readout in the " 
second direction of image pickup signals stored in the image storage 
portion, on the basis of the amount of movement in the second 
direction. 
[0024] 

The image storage control portion controls the readout of 
image pickup signals stored in the image storage portion in the first 
direction, on the basis of the amount of movement In the first 
direction. This enables correcting the image distortions caused by 
hand-shakes in the first direction. 
[0025] 

[Mode for Carrying out the Invention] 
[First Embodiment] 

With reference to Fig. 1, an image inputting apparatus 
according to a first embodiment includes an X-Y address type solid 
image pickup device 1 for converting optical images of an object into 
electrical signals, a pre-processing portion 2 which is connected to 
the solid Image pickup device 1 and performs pre-processing such as 
CDS (Correlated Double Sampling), AGC (Automatic Gain Control) 
and ADC (Analog to Digital Convert: A/D conversion) on the outputs 
from the solid image pickup device 1 to convert them into digital 
signals, a signal processing portion 3 which is connected to the 
pre-processing portion 2 and divides the digital signals resulted 
from the conversion by the pre-processing portion 2 into brightness 
signals and color signals, then performs white-balancing processing, 
gamma correction and the like on the aforementioned signals and 
converts the aforementioned signals into image signals of a proper 
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format (for example, YUV, RGB), and an I/F (Interface) portion 5 for 
use in inputting and outputting data to and from personal computers 
and the like. 
[0026] 

The image inputting apparatus further includes an image 
storage portion 4 which is connected to the signal processing portion 
3 and the I/F portion 5 and temporality stores image data, in cases 
where the image data is to be subjected to JPEG (Joint Photographic 
Experts Group) compression or MPEG (Moving Picture Experts 
Group) compression, In order to perform buffering of Image signals 
for changing the speed in cases where the transfer speed of image 
signals output from the signal processing portion 3 is not equal to 
the processing speed of the I/F portion 5, or in order to reduce the 
amount of data transferred through the I/F portion 5. Further, the 
Image inputting apparatus includes a timing generating portion 6 
which is connected to the solid image pickup device 1 and the 
pre-processing portion 2 and generates various types of pulses for 
driving the solid Image pickup device 1 and pulses required by the 
pre-processing portion 2, and an image storage control portion 7 
which is connected to the image storage portion 4 and controls the 
writing and the reading of data into and from the image storage 
portion 4. 
[0027] 

The image inputting apparatus further includes a hand-shake 
detecting portion 8 constituted by an angular-velocity sensor or the 
like mounted near the solid image pickup device 1, wherein the 
hand-shake detecting portion 8 detects hand-shake caused by a 
user during photographing and outputs hand-shake information 
according to the direction and the amount of the hand-shake. Also, 
the image inputting apparatus further includes a hand-shake 
correction value calculating portion 9 which is connected to the 
timing generating portion 6 and the hand-shake detecting portion 8, 
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wherein the hand-shake correction value calculating portion 9 
calculates correction data for correcting the movement due to the 
hand-shake, on the basis of the hand-shake Information output from 
the hand-shake detecting portion 8, on a pixel-by-pixel basis, and 
supplies the correction data to the timing generating portion 6, 
[0028] 

The I/F portion 5 is constituted by, for example, a PCI 
(Peripheral Component Interconnect) bus, a PC card, an USB 
(Universal Serial Bus), an IEEE (Institute of Electrical and 
Electronics Engineers) 1394 and the like. 
[0029] 

The hand-shake detecting portion 8 is not limited to an 
angular-velocity sensor. The solid image pickup device 1 includes 
an area sensor 11 including photoelectric conversion devices 
arranged in a two-dimensional shape, a vertical scanning portion 12 
which Is connected to the area sensor portion 11 and successively 
selects In the vertical direction the horizontal lines of the 
photoelectric conversion devices In the area sensor portion 11 on a 
llne-by-llne basis, a horizontal scanning portion 13 which Is 
connected to the area sensor portion 11 and successively selects in 
the horizontal direction Image signals output from the photoelectric 
conversion devices along the aforementioned lines, and an output 
circuit portion 4 for converting the pixel signals from the 
photoelectric conversion devices selected by the vertical scanning 
portion 12 and the horizontal scanning portion 13 into voltages and 
outputs the voltages. 
[0030] 

With reference to Fig. 2, the pixel structure of the area sensor 
portion 11 will be described. In this case, the number of the 
effective vertical lines of image signals output from the area sensor 
portion 1 is defined as N, and the number of the effective horizontal 
pixels is defined as M. The hand-shake correction value calculation 
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portion 9 calculates the amount of hand-shake generated between 
the first-selected line (j = l) and the last-selected line (j=N) In the 
area sensor portion 11, from the horizontal hand-shake information 
detected by the hand-shake detecting portion 8, and outputs, to the 
timing generating portion 6, the amount of hand-shake as correction 
data for correcting the movement due to the hand-shake. For 
example, the hand-shake correction value calculating portion 9 
determines, from calculation, the occurrence of leftward 
hand-shake corresponding to a pixels and outputs to the timing 
generating portion 6 correction data a for correcting the movement 
due to the hand-shake. 
[0031] 

With reference to Fig. 3, there will be described hand-shake 
correction processing in the timing generating portion 6. 
[0032] 

The timing generating portion 6 receives the horizontal 
correction data a from the hand-shake correction value calculating 
portion 9 and initializes a vertical line counter (j=l) (step SI). The 
timing generating portion 6 determines whether or not the 
horizontal correction data a is negative (step S2). If the correction 
data a Is positive, leftward hand-shake has occurred. If it Is 
negative, rightward hand-shake has occurred. 
[0033] 

In cases where the horizontal correction data a is positive or 
0 (leftward hand-shake or no hand-shake has occurred: the step S2 
results in No), an evaluation value x for use in determining the 
horizontal-pixel selection starting position is calculated on the basis 
of the following equation (1) (step S3). 
[0034] 

X=a*j/N ...(1) 

Here, in cases where no hand-shake has occurred, the 
horizontal correction data a is 0, and therefore, the horizontal-pixel 
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selection starting position is not changed. 
[0035] 

In cases where the horizontal correction data a is negative 
(rightward hand-shake has occurred: the step S2 results in Yes), the 
evaluation value x for use in determining the horizontal-pixel 
selection starting position is calculated on the basis of the following " 
equation (2) (step S4). 
[0036] 

X= a * j/N - a ...(2) 
In cases where the solid image pickup device 1 is a 
monochrome sensor (the step S5 results in Yes), In order to perform 
interpolation with a pixel closest to the pixel specified by the 
evaluation value x, the evaluation value x is rounded off to an 
integer value, and the rounded evaluation value x is substituted into 
the horizontal-pixel selection starting position p (step S6). 
[0037] 

In cases where the solid image pickup device 1 is a color 
sensor (the step S5 results In No), the area sensor portion 11 
includes a cojor filter, and therefore, it is necessary to change the 
selection starting position p depending on the filter arrangement. 
In this case, it Is assumed that the color filter having colors 
repeatedly arranged on two-pixel by two-pixel basis +n the 
horizontal and vertical directions, is used. 
[0038] 

The evaluation value x is divided by 2, then the divided 
evaluation value x is rounded off to an integer value, then the 
rounded value is doubled, and the doubled value is substituted into 
the horizontal-pixel selection starting position p (step S7). As a 
result, the selection starting position p is an even number, which 
ensures that the order of colors output from the solid image pickup 
device 1 is not changed by executing the following processing. This 
enables correctly performing color processing with the signal 
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processing portion 3. 
[0039] 

Next, a horizontal pixel counter is initialized (1 = 1) (step S8). 
Then, the (i+p)-th pixel on the j-th line is selected (step S9). 
Further, the horizontal pixel counter i is incrennented one by one 
(step SIO), and the processes at the steps S9 and SIC are repeated " 
until the number of output pixel data reaches (M-|a|) (step Sll). 
[0040] 

Further, the vertical line counter j is incremented one by one 
(step S12), and the processes at the steps S2 to S12 are repeated 
for all the effective vertical lines (step S13). 
[0041] 

As described above, the timing generating portion 6 controls 
to change the pixel selection starting position for the horizontal 
scanning portion 13, on the basis of the output from the hand-shake 
correction value calculating portion 9 and the position of the line 
selected by the vertical scanning portion 12. This enables 
correcting the image distortions caused by the horizontal 
hand-shake. 
[0042] 

Although the present processing reduces the number of 
effective horizontal pixels output from the solid image pickup device 
1 from M to (M-|a|), it is possible to overcome the problem by 
utilizing, as the solid Image pickup device 1, a solid image pickup 
device with a number of pixels greater than that of a required output 
screen size. 
[Second Embodiment] 

With reference to Fig. 4, an image inputting apparatus 
according to a second embodiment will be described. The image 
inputting apparatus according to the present embodiment employs a 
timing generating portion 26, instead of the timing generating 
portion 6, and also employs an image storage control portion 27, 




instead of the image storage control portion 7, in tlie liardware 
structure of the image inputting apparatus according to the first 
embodiment described with reference to Fig. 1. In the image 
inputting apparatus according to the first embodiment, the 
hand-shake correction value calculating portion 9 detects on a 
pixel-by-pixel basis the amount of movement, on the basis of 
hand-shake information supplied from the hand-shake detecting 
portion 8, and outputs to the timing generating portion 9 correction 
data for correcting the movement due to the hand-shake. However, 
in the present embodiment, the hand-shake correction value 
calculating portion 9 outputs correction data to the image storage 
control portion 27. The image storage control portion 27 controls 
the readout in the horizontal direction from the image storage 
portion 4 on the basis of the supplied correction data. 
[0043] 

The other hardware structure is the same as that of the first 
embodiment. Therefore, detailed description thereof is not 
provided herein. 
[0044] 

With reference to Fig. 5, there will be described hand-shake 
correction processing by the image storage control portion 27. 
[0045] 

The image storage control portion 27 receives horizontal 
correction data a from the hand-shake correction value calculating 
portion 9 and initializes a vertical line counter (j=l) (step S21). 
The image storage control portion 27 determines whether or not the 
horizontal correction data a is negative (step S22). If the 
correction data a is positive, leftward hand-shake has occurred. If 
It is negative, rightward hand-shake has occurred. 
[0046] 

In cases where the horizontal correction data a is positive or 
0 (leftward hand-shake or no hand-shake has occurred: the step 
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S22 results in No), an evaluation value x for use In determining the 
horizontal-pixel readout starting position is calculated on the basis 
of the aforementioned equation (l) (step S23). 
[0047] 

In cases where the horizontal correction data a is negative 
(rightward hand-shake has occurred: the step S22 results In Yes), 
the evaluation value x for use in determining the horizontal-pixel 
readout starting position Is calculated on the basis of the 
aforementioned equation (2) (step S24). 
[0048] 

The evaluation value x is rounded off to an integer value, and 
the rounded evaluation value x Is substituted into the 
horizontal-pixel readout starting position p (step S26). 
[0049] 

Next, a horizontal pixel counter is initialized (1=1) (step S28). 
Then, the (i+p)-th pixel on the j-th line is read out (step S29). 
Further, the vertical pixel counter i is incremented one by one (step 
S30), and the processes at the steps S29 and S30 are repeated until 
the number of output pixel data reaches (M-|a|) (step S31). 
[0050] 

Further, the vertical line counter j is Incremented one by one 
(step S32), and the processes at the steps S22-to S32 are repeated 
for all the effective vertical lines (step S33). 
[0051] 

As described above, the image storage control portion 27 
controls to change the horizontal pixel readout position, on the basis 
of the output from the hand-shake correction value calculating 
portion 9 and the position of the vertical line. This enables 
correcting the image distortions caused by the horizontal 
hand-shake. 
[0052] 

In the present embodiment, in order to simplify the 
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processing, the interpolation is performed witli the pixel data of a 
pixel closest to the original position of the pixel, in reading out 
image data from the image storage portion 4. However, the 
present invention Is not limited to the interpolation method, and the 
interpolation of pixel data may be performed using a linear 
interpolation method or a multinomial function. 
[Third Embodiment] 

In the aforementioned first and second embodiments, only 
horizontal hand-shake information detected by the hand-shake 
detecting portion 8 is employed for correcting image distortions 
caused by horizontal hand-shake. 
[0053] 

In the present embodiment, there will be described a method 
for correcting image distortions caused by vertical hand-shake, 
using vertical hand-shake information detected by the hand-shake 
detecting portion 8. 
[0054] 

The image inputting apparatus according to the present 
embodiment has the same hardware structure as that of the image 
inputting apparatus according to the second embodiment described 
with reference to Fig. 4. However, the hand-shake correction value 
calculating portion 9 and the image storage control portion 27 
according to the present embodiment execute processes which are 
partially different from those of the second embodiment, in order to 
address vertical hand-shake as well. The other hardware structure 
is the same as that of the second embodiment. Therefore, detailed 
description thereof is not provided herein. 
[0055] 

The hand-shake correction value calculating portion 9 
calculates the amount of horizontal hand-shake generated between 
the first-selected line (j = l) and the last-selected line (j = N) in the 
area sensor portion 11, from the horizontal hand-shake information 
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detected by the hand-shake detecting portion 8. Further, the 
hand-shake correction value calculating portion 9 calculates the 
amount of vertical hand-shake, from the vertical hand-shake 
information detected by the hand-shake detecting portion 8. The 
hand-shake correction value calculating portion 9 outputs, to. the 
image storage control portion 27, these amounts of hand-shakes as " 
correction data for correcting the movement due to the hand-shakes. 
For example, the hand-shake correction value calculating portion 9 
determines, from calculation, the occurrence of leftward 
hand-shake corresponding to a pixels and the occurrence of upward 
hand-shake corresponding to b lines and outputs, to the image 
storage control portion 27, horizontal correction data a and vertical 
correction data b for correcting the movement due to the 
hand-shakes. 
[0056] 

Accordingly, the image storage control portion 27 controls the 
vertical and horizontal readout from the image storage portion 4. 
[0057] 

With reference to Fig. 6, there will be described hand-shake 
correction processing by the Image storage control portion 27. 
[0058] 

The Image storage control portion 27 receives the horizontal 
correction data a and the vertical correction data b, from the 
hand-shake correction value calculating portion 9, and initializes a 
vertical line counter (j = l) (step S34). 
[0059] 

The image storage control portion 27 calculates an evaluation 
value y for use in determining the vertical line to be read out, on the 
basis of the following equation (3). 
[0060] 

y = (N + b) * j/N ...(3) 

Next, in order to perform interpolation with the pixel data on 
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the line position resulted from hand-shake correction and a line 
closest to the line, the evaluation value y Is rounded off to an Integer 
value, and the rounded evaluation value y Is substituted into the 
vertical readout line q (step S35). 
[0061] 

If the vertical readout line q exceeds the number of the " 
effective lines N (the step S36 results in Yes), the readout of data 
from the image storage portion 4 ends. If the vertical readout line 
q Is equal to or less than the number of the effective lines N (the step 
S36 results in No), the image distortions caused by the horizontal 
hand-shake Is corrected. In a same way as that in the second 
embodiment (steps S22 to S31). 
[0062] 

However, the present embodiment Is different from the 
second embodiment In that the j-th line is selected as a line for 
reading out data (step S29 in Fig. 5) In the second embodiment, 
while the q-th line is selected (step S37 In Fig. 6) in the present 
embodiment. By changing the vertical readout line according to 
the vertical correction data b, as described above, the Image 
distortions caused by the vertical hand-shake can be corrected. 
[0063] 

Thereafter, the vertical line counter j is Incremented one by 
one (step S32), and the processes at the steps S35 to S32 are 
repeated for all the effective vertical lines (step S33). 
[0064] 

As described above, the image storage control portion 27 
controls to change the vertical-line readout position and the 
horizontal-pixel readout position from the image storage portion 4, 
on the basis of the output from the hand-shake correction value 
calculating portion 9. This enables correcting the Image distortions 
caused by the vertical and horizontal hand-shakes. 
[0065] 
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Also, it is possible to correct the image distortions caused by 
horizontal hand-shake, according to the same method as that of the 
first embodiment. 
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FIG. 4 



12 11 ] 14 2 



zn 



13 



T 



1 



L J. 



'26 











— ^ 





mm 



-27 



2) PRE-PROCESSING PORTION 

3) SIGNAL PROCESSING PORTION 

4) IMAGE STORAGE PORTION 

5) I/F PORTION 

8) HAND-SHAKE DETECTING PORTION 

9) HAND-SHAKE CORRECTION VALUE CALCULATING PORTION 

11) AREA SENSOR PORTION 

12) VERTICAL SCANNING PORTION 

13) HORIZONTAL SCANNING PORTION 

14) OUTPUT CIRCUIT PORTION 

26) TIMING GENERATING PORTION 

27) IMAGE STORAGE CONTROL PORTION 



-22- 



FIG. 5 




S23- ^ x*-a -j/N i 824'- ^x«Ta'jVN-ai 




S33^ 



No 




START 

S21) a^HORIZONTAL CORRECTION DATA 

S26) p^x; X HAS BEEN ROUNDED-OFF TO AN INTEGER VALUE 
S29) SELECT (i + p)-TH DATA ON j-TH LINE 
END 



-23- 



FIG. 6 



S35' 



r-{N+b) • j/N 




823- ^ x<-a'j77r| S24-vjx«-a'J/N~aJ 



— . T — 



^S37 




START 

534) a^HORIZONTAL CORRECTION DATA 
b^VERTICAL CORRECTION DATA 

535) y^(N + b)*j/N 



-24- 



p-y; y HAS BEEN ROUNDED-OFF TO AN INTEGER VALUE 
S26) p^x; X HAS BEEN ROUNDED-OFF TO AN INTEGER VALUE 
S37) SELECT (i + p)-TH DATA ON q-TH LINE 

END 



-25- 



FIG. 7 
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